Amendments to the claims: 

This listing of claims replaces all prior versions and listings of claims in the 
application. 
Listing Of Claims: 

1 . (Currently amended) A processor-based method for determining difficulty 
measures for training cases used in developing a solution to a problem, comprising: 

providing a set of training cases having respectively associated difficulty 
measures; 

operating a candidate solution on a particular training case; 

determining a performance measure of the candidate solution operating on the 
s e l e cted particular training case; 

determining a credibility rating of the candidate solution, the credibility rating 
indicating a degree to which the performance measure is representative of the difficulty 
measure of the particular training case proficioncv lovol of tho cand i dat e solution at 
so l ving tra i n i ng cas e s ; and 

modifying the difficulty measure of the particular training case based on the 
performance measure of the candidate solution operating on the particular training case 
and the credibility rating of the candidate solution. 

2. (Original) The method of claim 1 , wherein determining the credibility rating 
comprises: 

selecting one or more training cases from the set of training cases based on the 
difficulty measures of the one or more training cases; 

determining performance measures of the candidate solution operating on each 
of the one or more training cases; and 

computing the credibility rating based on the performance measures of the 
candidate solution operating on each of the one or more training cases. 

3. (Original) The method of claim 2, wherein the one or more training cases does not 
include the particular training case. 
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4. (Original) The method of claim 1 , wherein providing the set of training cases having 
respectively associated difficulty measures comprises initializing a difficulty measure of 
each training case in the set of training cases to a predetermined value. 

5. (Original) The method of claim 4, wherein the predetermined value is a maximum 
value. 

6. (Original) The method of claim 1 , wherein: 

providing the set of training cases comprises associating each training case in 
the set of training cases with a target output; 

operating the candidate solution on the particular training case comprises 
obtaining an output from the candidate solution operating on the particular training 
case; and 

determining the performance measure of the candidate solution operating on the 
particular training cases comprises comparing the candidate solution output to a target 
output of the particular training case. 

7. (Original) The method of claim 6, wherein comparing the candidate solution output 
to the target output of the particular training case comprises calculating a value 
corresponding to a deviation between the candidate solution output and the target 
output of the particular training case. 

8. (Original) The method of claim 1 , wherein modifying the difficulty measure of the 
particular training case comprises modifying the difficulty measure based on a weighted 
average of the performance measure and a previous value of the difficulty measure. 

9. (Original) The method of claim 8, wherein a weight of the weighted average is 
based on the credibility rating and a base learning rate. 

10. (Original) The method of claim 1, wherein modifying the difficulty measure 
comprises maintaining the difficulty measure within a predetermined interval. 
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1 1 . (Withdrawn) A processor-based method for solving a problem, comprising: 

providing a set of candidate solutions; 

providing a set of training cases having respectively associated difficulty 
measures; 

operating a candidate solution on a particular training case; 

determining a performance measure of the candidate solution operating on the 
particular training case; 

determining a credibility rating of the candidate solution, the credibility rating 
indicating a proficiency level of the candidate solution at solving training cases; 

modifying the difficulty measure of the particular training case based on the 
performance measure of the candidate solution and the credibility rating of the 
candidate solution; 

evaluating the candidate solution; and 

modifying the set of candidate solutions based on the evaluation. 

12. (Withdrawn) The method of claim 1 1 , wherein: 

providing the set of training cases comprises respectively associating the training 
cases in the set of training cases with target outpuis; 

operating the candidate solution on the particular training case comprises 
obtaining an output from the candidate solution operating in the context of the particular 
training case; and 

determining the performance measure of the candidate solution operating on the 
particular training case comprises comparing the candidate solution output to a target 
output of the particular training case. 

13. (Withdrawn) The method of claim 1 1 , wherein evaluating the candidate solution 
comprises: 

selecting one or more training cases based on the difficulty measures of the one 
or more training cases; and 
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operating the candidate solution on each of the one or more selected training 

cases. 

14. (Withdrawn) The method of claim 13, wherein selecting the one or more training 
cases comprises biasing selection of the one or more training cases towards more 
difficult training cases in the set of training cases. 

15. (Withdrawn) The method of claim 13, wherein selecting each of the one or more 
training cases comprises: 

randomly selecting a subset of training cases from the set of training cases; and 
selecting a training case from the subset of training cases based on the difficulty 
measure of the selected training case. 

16. (Withdrawn) The method of claim 15, wherein selecting the training case 
comprises selecting the training case having a relatively high difficulty measure in 
comparison with other training cases in the subset of training cases. 

1 7. (Withdrawn) The method of claim 1 3, wherein selecting each of the one or more 
training cases comprises: 

selecting a subset of training cases from the set of training cases based on the 
difficulty measures of members of the subset; and 
selecting a training case from the subset. 

18. (Withdrawn) The method of claim 13, wherein selecting each of the one or more 
training cases comprises selecting a training case based on a probability computed as 
a function of a difficulty measure of the selected training case and the difficulty 
measures of members of the set of training cases. 

1 9. (Withdrawn) The method of claim 1 1 , wherein determining the performance 
measure of the candidate solution comprises calculating a value corresponding to a 
deviation between an output of the candidate solution and the target output. 
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20. (Withdrawn) The method of claim 1 1 , wherein modifying the set of candidate 
solutions comprises modifying one or more particular candidate solutions in the set of 
candidate solutions. 

21 . (Withdrawn) The method of claim 1 1 , wherein modifying the set of candidate 
solutions comprises forming a new set of candidate solutions. 

22. (Withdrawn) A system, comprising: 

an input unit configured to provide a set of training cases and a set of candidate 
solutions, each training case having an associated difficulty measure; and 

a processor configured to determine a credibility rating of a candidate solution 
and a performance measure of the candidate solution operating on a particular training 
case, the processor further configured to modify a difficulty measure of the particular 
training case based on the performance measure and the credibility rating, 

wherein the credibility rating indicates a proficiency of the candidate solution at 
solving training cases. 

23. (Withdrawn) The system of claim 22, wherein the processor is configured to 
select one or more training cases from the set of training cases based on difficulty 
measures of the one or more selected training cases, determine performance 
measures of the candidate solution operating on each of the one or more selected 
training cases, and compute the credibility rating based on the performance measures 
of the candidate solution operating on each of the one or more training cases. 

24. (Withdrawn) The system of claim 22, wherein the processor is configured to 
obtain an output from the candidate solution operating on the particular training case, 
compare the candidate solution output and a target output of the particular training 
case, and determine the performance measure of the candidate solution based on the 
comparison. 
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25. (Withdrawn) The system of claim 22, wherein the processor is configured to 
modify the difficulty measure of the particular training case based on a weighted 
average of the performance measure and a previous value of the difficulty measure. 

26. (Withdrawn) The system of claim 22, wherein the processor is further configured 
to evaluate the candidate solution and modify the set of candidate solutions based on 
the evaluation. 

27. (Withdrawn) The system of claim 26, wherein the processor is configured to 
select one or more training cases based on the difficulty measures of the one or more 
training cases, operate the candidate solution on the one or more training cases, 
determine a performance measures of the candidate solution operating on the one or 
more selected training cases, and evaluate the candidate solution based on the 
performance measures. 

28. (Withdrawn) The system of claim 27, wherein the processor is configured to bias 
selection of the one or more training cases towards training cases that are more difficult 
to solve than other training cases in the set of training cases. 

29. (Withdrawn) A system for determining training case difficulty measures, 
comprising: 

means for providing a set of training cases having respectively associated 
difficulty measures; 

means for operating a candidate solution on a particular training case; 

means for determining a performance measure of the candidate solution 
operating on the selected training case; 

means for determining a credibility rating of the candidate solution, the credibility 
rating indicating a proficiency of the candidate solution at solving training cases; and 

means for modifying the difficulty measure of the particular training case based 
on the performance measure of the candidate solution operating on particular training 
case and the credibility rating of the candidate solution. 
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30. (Withdrawn) An article of manufacture for determining a solution to a problem, 
comprising: 

a computer-readable medium configured with instructions for causing a computer 
to perform the method of, 

providing a set of candidate solutions; 

providing a set of training cases having respectively associated difficulty 
measures; 

operating a candidate solution on a particular training case; 

determining a performance measure of the candidate solution operating on the 

particular training case; 

determining a credibility rating of the candidate solution, the credibility rating 
indicating a proficiency of the candidate solution at solving training cases; 

modifying the difficulty measure of the particular training case based on the 
performance measure of the candidate solution and the credibility rating of the 
candidate solution; 

evaluating the candidate solution; and 

modifying the set of candidate solutions based on the evaluation. 



8 



